group difference divided by the within-group standard deviation.

Small samples will not be able to produce significant results unless the effect size is very large.

Conversely, statistical tests using extremely large samples including many thousands of

participants are almost always statistically significant unless the effect size is near zero. In

epidemiological studies, which often involve hundreds of thousands of subjects, statistical tests

tend to produce extremely small (and therefore extremely significant) p values, even when the

effect size is so small that it’s of no practical importance (meaning it is clinically insignificant).

© John Wiley & Sons, Inc.

FIGURE 3-2: The power of a statistical test increases as the sample size and the effect size increase.

Power versus effect size, for various sample sizes: For all statistical tests, power always

increases as the effect size increases, if other variables including the α level and sample size are

held constant. This relationship is illustrated in Figure 3-3. “N” is the number of participants in

each group.